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CLAIMS 

[Claim(s)] 

[Claim 1]A modeling device comprising: 

A position measurement means which measures a position of the tip part concerned and a 
supporter about a model object which has a joint linked to two or more tip part and these tip 
parts, such as a fingertip, and a supporter which supports the tip part concerned via said joint. 
A model data generating means which generates model data of said model object based on 
position information on each tip part measured by this position measurement means and a 
supporter. 

It has a model data output means which outputs model data generated by this model data 
generating means to external instruments, such as a displaying means, A restraint condition 
storage part said model data generating means remembered a constraint about flexibility and 
angle of rotation of said joint which were beforehand defined according to an example of said 
model object of operation to be. 

A geometric model synchronizer which compounds a geometric model for every tip part 
generated by geometric model generation part which computes a position of the tip part 
concerned and the tip part concerned, and an angle of a linked joint, and this geometric model 
generation part for every tip part based on a constraint stored in this restraint condition storage 
part, and position information on said tip part. 

[Claim 2]The modeling device comprising according to claim 1: 

The 1st constraint storage area that memorizes the 1st constraint as which said restraint 
condition storage part determined a relation of an angle of each joint beforehand about one tip 
part and two or more linked joints. 

A full-size degree calculation function in which said geometric model generation part computes 
from a supporter a relation of an angle of each joint which exists by a tip part based on a 
position of a supporter measured by said position measurement means, and a position of said tip 
part when the 1st constraint is imposed about a tip part which generates a model. 

[Claim 3] It has the 2nd constraint storage area that memorizes the 2nd constraint fixed at an 
angle which gave flexibility only to a joint with said restraint condition storage part specific about 
one tip part and two or more linked joints, and was beforehand defined about other joints, Said 
geometric model generation part, When the 2nd constraint is imposed about a tip part which 
generates a model, by said position measurement means. Compute an angle of a joint specified 
by the 2nd constraint of the inner above of each joint which exists by a tip part from the 
supporter concerned based on a position of a measured supporter, and a position of said tip part, 
and. The modeling device according to claim 1 or 2 provided with a specific joint angle 
calculation function judged to be the angle beforehand defined about other joints. 
[Claim 4]Said restraint condition storage part, It has the 3rd constraint storage area that 
memorizes a tip part and a joint used as a standard by which an angle of each joint linked to a 
position of an unmeasured tip part by which a position is not measured by the inner 
aforementioned position measurement means of a tip part of said model object, and the 
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unmeasured tip part concerned was defined beforehand as the 3rd constraint. Said geometric 
model generation part. When said 3rd constraint is imposed on an unmeasured tip part which 
generates a model, judge a position which follows a position of a tip part used as a standard of 
the unmeasured tip part concerned to be a position of the unmeasured tip part concerned, and. 
The modeling device according to claim 1, 2, or 3 provided with an unmeasured partial calculation 
function computed based on an angle of the unmeasured tip part concerned, a tip part which 
serves as said standard about an angle of each linked joint, and each linked joint. 
[Claim 5]A position metering device which measures a position of the tip part concerned and a 
supporter about a model object which has two or more tip parts, such as a fingertip, these tip 
parts, a linked joint, and a supporter which supports the tip part concerned via said joint, An 
arithmetic unit which generates model data of said model object based on position information on 
each tip part which is connected with this position measurement means and inputted from the 
position measurement means concerned, A modeling system provided with constraint memory 
storage which memorized a constraint about flexibility and angle of rotation of said joint which 
were beforehand defined according to an example of a model object of operation is used. A 
constraint reading command to which are the storage which memorized a program for modeling 
data generation for generating modeling data in real time, and a constraint stored in said 
restraint condition storage part is made to read for every kind of said tip part, Geometric model 
generation instructions which make a position of the tip part concerned and the tip part 
concerned, and an angle of a linked joint compute for every tip part based on a constraint read 
according to this constraint reading command, and position information on said tip part, A 
storage which memorized a program for modeling data generation provided with geometric model 
composition instructions which make a geometric model for every tip part generated according 
to these geometric model generation instructions compound. 

[Claim 6]The 1st constraint data that is provided with the following and said restraint condition 
storage part makes equal [ an angle of each joints of two or more ] about an index finger model 
and a middle finger model, The 2nd restricted data fixed at an angle which gave flexibility only to 
a joint which hits the root of the thumb concerned about a thumb model, and was beforehand 
defined about other joints, It has a restricted data storage area provided with the 3rd restricted 
data that defined flattery form of making operation of said middle finger model following about a 
third finger model and a digitus-minimus model, A modeling device, wherein said geometric model 
generation part is provided with an approximation shape calculation function which computes 
shape of each of said fingertip model based on one ingredient in three dimensional coordinates of 
said fingertip position, and said each restricted data. 

A position measurement means which a fingertip position of the thumb, an index finger, and the 
middle finger is measured among the digiti-manus point positions concerned by using shape of a 
hand of an operator as a model object, and computes three dimensional coordinates of the 
fingertip position concerned. 

A model data generating means which generates geometric model data of a hand which consists 
of a fingertip model which has a joint based on position information on each tip part measured by 
this position measurement means, and the back of a hand. 

It has a model data output means which outputs geometric model data of a hand generated by 
this model data generating means to external instruments, such as a displaying means, A 
restraint condition storage part said model data generating means remembered a constraint 
about flexibility and angle of rotation of each joint of each finger beforehand defined according to 
an example of said fingertip model of operation to be. 

A model-synthesis part which compounds a geometric model generation part which generates a 
fingertip model for every finger based on a constraint stored in this restraint condition storage 
part, and position information on said tip part, and a fingertip model for every tip part generated 
by this geometric model generation part and a shell model of said hand. 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the lnvention]This invention relates to a modeling device and relates to the modeling 
device which generates a geometric model in real time especially by using operation of human 
being's hand, a motion of an animal, etc. as a model object. 

[0002]The model object which is the target of geometric model generation by this invention is 
provided with the joint connected with a tip part and this tip part, and the supporter which 
supports a tip part via this joint. If it is a thing which has this tip part and joint, various things, 
such as a living body and an object on the imagination to an inorganic substance pan, can be 
used as a model object. That is, this invention is suitable for generation of the geometric model 
of 4 foot-actuation things, such as a geometric model of the hand which uses a fingertip as a tip 
part, and a horse which uses the point of a leg as a tip part, the geometric model of the trees 
which use a leaf tip as a tip part, and the geometric model of a folding umbrella. 
[0003]As an applicable field of this invention, it is applicable to the general computer graphics 
technology in a virtual reality. The simulation of an operation and the application to a game can 
be considered. 
[0004] 

[Description of the Prior Art]Conventionally, displaying operation of a hand on a display in real 
time is performed. For example, the technique of operating the object in virtual space is 
indicated by JP,H10-207617,A by the same applicant using the three-dimensional input 
manipulator. 
[0005] 

[Problem(s) to be Solved by the Invention] However, in the example indicated by the above- 
mentioned gazette, irrespective of the shape of the hand under operation, the model of the hand 
displayed on a display lengthened the index finger, and actually showed it in the shape in the 
state where other fingers were grasped. Then, there was inconvenience that the state of the 
hand displayed that the shape of the model of a hand is constant on the state and display of a 
hand of an operator will be different to being what gives the reaction force from a virtual model 
to three fingertips, for example. 

[0006]Although it is considered as the technique of carrying out the modeling of the state of a 
hand and motion capture is mentioned, this does not display real time. Although there is a 
method of attaching a distortion sensor to the finger joint part of a glove, and detecting the bend 
condition of each finger in real time, there is inconvenient [ that a motion of a finger is restrained 
not a little ]. 
[0007] 

[Objects of the lnvention]This invention improves the inconvenience which the starting 
conventional example has, and it sets as the purpose providing the modeling device which can 
generate the geometric model of the model object concerned in real time, securing natural 
operation of the model object in which shape becomes settled by the tip part and a joint 
especially. 
[0008] 
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[Means for Solving the Problemjln this invention, in order not to interfere in natural operation of 
a model object but to know a state of a model object, only a tip part and a supporter perform 
position Measurement Division, And it emulates from a position of a tip part, and a position of a 
supporter about angle of rotation, a direction, etc. of a joint which connects each tip part and a 
supporter, without carrying out direct measuring. Since there are few measure points, a 
constraint is given [ as opposed to / at this time / a total of flexibility of each joint ] to angle of 
rotation of each joint according to that difference. What is made equal [ angle of rotation of a 
joint connected with one tip part for example ] as a constraint, a relation which it is twice the 
relation between a joint angle of a certain joint and a joint angle of other joints of this, or 
numerals of plus or minus reverse, etc. correspond. 

[0009]A position measurement means which measures a position of the tip part concerned, and a 
position of a supporter which supports the tip part concerned via said joint about a model object 
which specifically has the joint linked to two or more tip part and these tip parts, such as a 
fingertip, by this invention, A model data generating means which generates model data of said 
model object based on position information on each tip part measured by this position 
measurement means and a supporter, It has a model data output means which outputs model 
data generated by this model data generating means to external instruments, such as a 
displaying means. And a restraint condition storage part a model data generating means 
remembered a constraint about flexibility and angle of rotation of said joint which were 
beforehand defined according to an example of said model object of operation to be, A geometric 
model generation part which computes a position of the tip part concerned and the tip part 
concerned, and an angle of a linked joint for every tip part based on a constraint stored in this 
restraint condition storage part, and position information on said tip part, Composition of having 
had a geometric model synchronizer which compounds a geometric model for every tip part 
generated by this geometric model generation part is taken. It is going to attain the purpose 
which this mentioned above. 

[0010] A position measurement means measures a position of each tip part of a model object, 
and a position of a supporter. And in a model data generating means, a geometric model 
generation part generates model data based on position information on this tip part, and position 
information on a supporter. Since there is few position information measured to a joint number at 
this time, a constraint about flexibility and angle of rotation of a joint is used. This constraint is 
beforehand stored in a restraint condition storage part, and a geometric model generation part 
reads and uses a constraint stored in this restraint condition storage part. Geometric model data 
of a complicated model object which has a joint by little position information by using this 
constraint is generated. 
[0011] 

[Embodiment of the lnvention]Hereafter, an embodiment of the invention is described with 
reference to Drawings. The modeling device by this embodiment is provided with the following. 
The position measurement means 2 which measures the position of the tip part concerned and a 
supporter about the model object 22 which has the joint linked to two or more tip part and these 
tip parts, such as a fingertip, and a supporter which supports the tip part concerned via said 
joint. 

The model data generating means 1015 which generates the model data of said model object 
based on the position information on each tip part measured by this position measurement 
means 2 and a supporter. 

The model data output means 1011 which outputs the model data generated by this model data 
generating means 1015 to external instruments, such as a displaying means. 
The restraint condition storage part 1012 the model data generating means 1015 remembered 
the constraint about the flexibility and angle of rotation of said joint which were beforehand 
defined according to the example of said model object 22 of operation to be, The geometric 
model generation part 1013 which computes the position of the tip part concerned and the tip 
part concerned, and the angle of the linked joint for every tip part based on the constraint 
stored in this restraint condition storage part 1012, and the position information on said tip part, 
It has the geometric model synchronizer 1014 which compounds the geometric model for every 
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tip part generated by this geometric model generation part 1013. 

[0012]The model object is made into the shape of people's hand in the example shown in drawing 
1. For this reason, the position measurement means 2 measures the position of the fingertip 
which is a tip part of a hand. In the example shown in drawing 1, the marks 21 B thru/or 21 F used 
as the feature on Image Processing Division are given to the fingertip, and the position of each 
mark is measured by the principle of triangulation. For this reason, the position measurement 
means 2 is provided with the following. 

The imaging camera 23 which photos the hand containing a fingertip. 
The light source 24 which irradiates a hand with measurement light. 
In this case, the geometric model generation part 1015 computes the distance to each mark 
based on the angle which a light source and the imaging camera 23 accomplish, and the image 
pickup position of each mark, and it is provided with the image processing function which 
changes the distance to each of this mark into the coordinates in the coordinate system in a 
measuring object. This image processing function changes the coordinate value of each point in a 
camera coordinate system into the coordinate value in object frame. By this image processing 
function, a three-dimensional range finder can be used as a three dimensional position metering 
device. Although the three dimensional position metering device by Image Processing Division is 
adopted as the position measurement means 2, it may be made to use other sensors which 
detect the amount of displacement and speed of not only this but a fingertip in the example 
shown in drawing 1. 

[001 3]If the position of each fingertip which is a tip part which stands in a row at a joint is 
measured, the geometric model generation part 1015 will generate the model data of a hand 
based on the position information on each of this fingertip. At this time, the geometric model 
generation part 1015 uses various constraints, in order to emulate the shape of a hand. That is, 
detecting position information only by a fingertip, since the shape becomes settled at three 
joints, the finger cannot specify angle of rotation of each joint by single position information. For 
this reason, a constraint is given according to the character of operation of each finger. 
[0014]Drawing 2 is a block diagram showing the relation of the constraint data and each function 
of the geometric model generation part 1014 which are stored in the restraint condition storage 
part 1012. The restraint condition storage part 1012 is provided with the 1st constraint storage 
area that memorizes the 1st constraint 1021 that defined the relation of the angle of each joint 
beforehand about one tip part and two or more linked joints first. Assumption that the 1st 
constraint has one tip part and the same angle of two or more linked joints, for example, and the 
assumption about the size ratio of each angle correspond. Such assumption is good to set 
according to the character of a model object. For example, if it is an index finger of people's 
hand, though angle of rotation of each joint is the same, the geometric model to generate will be 
in a natural state. The size ratio of each angle is the 2nd joint when the 1st joint is set to a 
[rad], for example, a/2 The constraint set to [rad] is meant. 

[001 5] When the 1st constraint is imposed according to this 1st constraint about the tip part in 
which the geometric model generation part 1014 generates a model, It has the full-size degree 
calculation function 1016 which computes from a supporter the relation of the angle of each joint 
which exists by a tip part based on the position of the supporter measured by the position 
measurement means 2, and the position of said tip part. Namely, in the example shown in drawing 
1, angle of rotation of three joints is computed by using a fingertip as a tip part using two 
position information. 

[0016]The restraint condition storage part 1012 is provided with the 2nd constraint storage area 
that memorizes the 2nd constraint fixed at the angle which gave flexibility only to the joint 
specific about one tip part and two or more linked joints next, and was beforehand defined about 
other joints. Even if this 2nd constraint is a case where two or more joints which support a tip 
part exist, it is conditions of restraining at the angle which gave flexibility only to the specific 
joint and was beforehand defined about other joints. 

[001 7] When the 2nd constraint is imposed according to this 2nd constraint about the tip part in 
which the geometric model generation part 1013 generates a model, Based on the position of the 
supporter measured by the position measurement means 2, and the position of said tip part, The 
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angle of the joint specified by the 2nd constraint of the inner above of each joint which exists by 
a tip part from the supporter concerned is computed, and it has the specific joint angle 
calculation function 1017 judged to be the angle defined beforehand about other joints. The 
number of specific joints becomes settled according to the number of measure points, the 
number of joints, and the number of flexibility. It is good to use this 2nd constraint for generation 
of the geometric model of the thumb in the example shown in drawing 1. Since angle of rotation 
of three joints is specified using the position information on two points, the fingertip of the 
thumb, and a supporter, the fingertip side specifies angle of rotation of two joints beforehand, 
and after limiting flexibility for the root of the thumb in the one direction, angle of rotation is 
computed here based on the measured position information. 

[0018]The restraint condition storage part 1012 further, By the inner aforementioned position 
measurement means 2 of the tip part of the model object 22. It has the 3rd constraint storage 
area 1023 that memorizes the tip part and joint used as the standard by which the angle of each 
joint linked to the position of the unmeasured tip part by which a position is not measured, and 
the unmeasured tip part concerned was defined beforehand as the 3rd constraint. This is a 
constraint for not measuring a position about the tip of a digitus minimus, but, for example, 
making the shape of a digitus-minimus model, and operation follow the shape of the model'of the 
third finger, and operation. This 3rd constraint is provided with the following. 
The flattery conditions of the position which specifically computes the amount of displacement 
of a digitus-minimus fingertip based on the amount of displacement of a third finger fingertip. 
The flattery conditions of the angle which makes angle of rotation which each joint of the third 
finger computed angle of rotation of each joint of a digitus minimus. 

[0019]Said geometric model generation part, When said 3rd constraint is imposed on the 
unmeasured tip part which generates a model, judge the position which follows the position of 
the tip part used as the standard of the unmeasured tip part concerned to be a position of the 
unmeasured tip part concerned, and. The modeling device according to claim 1, 2, or 3 provided 
with the unmeasured partial calculation function computed based on the angle of the 
unmeasured tip part concerned, the tip part which serves as said standard about the angle of 
each linked joint, and each linked joint. 

[0020] Drawing 3 is a block diagram showing the composition of the hardware resources of this 
embodiment. The model data generating means 1015 and the model data output means 1011 
which are shown in drawing 1 are realizable by computers (arithmetic processing unit), such as a 
workstation. The arithmetic processing unit 1010 is provided with the following. 
Measurement Division I/F1031 which receives the image data picturized by the imaging camera 
23 which constitutes a part of position measurement means. 

CPU1037 which performs processing which the coordinate value of the fingertip of each finger is 
computed based on this image data, and generates the geometric model of each finger based on 
the coordinate value concerned. 
RAM1038 used as the main memory of this CPU1037. 

ROM1039 which memorized the procedure at the time of starting of the arithmetic processinq 
unit 1010, etc. " ' 

[0021]lnput-and-output I/F1033 which manages input and output with the hard disk 1034 and 
disk drive 1035 grade according to control according [ the arithmetic processing unit 1010 ] to 
CPU1037 further, The disk drives 1035, such as a CD-ROM drive which reads the program data 
concerned from storages, such as CD-ROM in which the program for modeling data generation 
was stored, When this program for modeling data generation is installed, it has the hard disk (HD) 
1034 which memorizes the file which constitutes this program. CPU1037 is executing this 
program for modeling data generation, and functions as the model data generating means 1015 
shown in drawing 1. Constraint data as shown in drawing 2 is stored in the hard disk 1034, and it 
functions on it as constraint memory storage. 

[0022]The arithmetic processing unit 1010 is provided with the display interface 1032 used as an 
interface with the display which displays again the geometric model data generated by CPU1037 
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on a display. Display I/F1032 has RAM for a display, is performing depiction processing of a 
polygon, etc. and functions as the model data output means 1011 shown in drawing 1. 
[0023]The program for modeling data generation stored in the storage is provided with the 
following. 

The constraint reading command to which the constraint stored in the hard disk 1034 is made to 
read for every kind of said tip part as instructions for operating said CPU1037. 
Geometric model generation instructions which make the position of the tip part concerned and 
the tip part concerned, and the angle of the linked joint compute for every tip part based on the 
constraint read according to this constraint reading command, and the position information on 
said tip part. 

Geometric model composition instructions which make the geometric model for every tip part 
generated according to these geometric model generation instructions compound. 

[0024]When calling it "instructions to operate" here, either or the both sides of the instructions 
which operate an arithmetic processing unit only by each instructions, and the instructions which 
operate the computer concerned depending on other programs, such as an operating system 
beforehand stored in the arithmetic unit, is included. For example, in the example shown in 
drawing 3, geometric model generation instructions read the constraint data stored in the hard 
disk depending on the file input output function of an operating system. Therefore, geometric 
model generation instructions are instructions which read to an operating system and hand over 
the target file name, for example. Thus, it is a storage which memorizes the program for 
analytic-model generation concerned, and "instructions which reads to an operating system and 
hands over the target file name" may be stored in the storage of the use which conveys the 
program concerned to a user, for example. This becomes settled in a relation with the operating 
system etc. of the computer which it is going to operate. 

[0025]Lighting Sub-Division 24 irradiates with measurement light the hand 22 which is a model 
object. The imaging camera 23 receives the catoptric light from this hand 22, and inputs shade 
image data into Measurement Division I/F1031. CPU1037 computes the distance of the 
fingertips 21 B thru/or 21 F from this shade image data first. For example, based on the angle of 
Lighting Sub-Division 24 and the imaging camera 23 to accomplish, the principle of triangulation 
generates a depth map from a shade image, and it may be made to compute the coordinates of 
each fingertip further by extracting the portion according to the radius of circle of the fingertip 
among depth maps. Whether each radius-of-circle portion is equivalent to which finger gives 
which finger each radius of circle is by the initial condition, and it is specified after next 
Measurement Division of which finger it is a fingertip according to the amount of displacement of 
each radius-of-circle portion from the last measuring position. A respectively individual mark is 
given to the fingertip instead of extraction of a radius-of-circle portion, and it may be made to 
specify according to the image recognition. In order to give reaction force to a fingertip, the 
operation input means 2A in working example mentioned later is used. 
[0026]lf the coordinates of each fingertip are specified, CPU1037 will read the constraint 
according to each finger from the hard disk 1034. The 2nd constraint 1022 is read in calculation 
of a thumb model. And angle of rotation of the root of the thumb is computed. It attaches by a 
digitus minimus from an index finger, and the 1st constraint is read. And the angle of each joint 
of one finger is a premise made equal, and computes a joint angle for every finger. Then, from 
the fingertip position of each finger, and the angle of a joint, CPU1037 generates the link used as 
the skeleton of the model of a hand, forms the cylindrical shape which makes this link a medial 
axis, and forms the shape defined beforehand in accordance with the direction of a finger about 
the back of a hand. Thereby, the geometric model of a hand is formed. A geometric model of this 
sort is displayed on a display via display I/F1032. By repeating Measurement Division of these 
fingertip coordinates, and generation of a geometric model with a predetermined sampling period, 
the modeling of the operation of the hand which is a model object can be carried out in real time. 

[0027] 

[Example]Next, working example of this invention is described with reference to Drawings. At this 
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example, the modeling of the shape of a hand is carried out by measuring three positions of the 
fingertip position of the thumb, an index finger, and the middle finger. In the index finger model 
and middle finger model which are geometric models of an index finger and the middle finger, 
formed data is generated using the 1st constraint that each joint angle is the same. Flexibility is 
given only to the joint of the root of the thumb about the thumb, and the 2nd constraint made 
into the angle defined beforehand is used about other joints. About a third finger model and a 
digitus-minimus model, the 3rd constraint made to follow the formed data of the middle finger is 
used. To a motion of the hand of the operator which is a measuring object, real time can be 
followed, the geometric model data of the hand that this has five tip parts and a finger which has 
three joints, respectively can be generated, and it can display on external instruments, such as a 
display. 

[0028]ln this example, the geometric model of a hand is arranged in virtual space, it displays on a 
display, and reaction force is given to a fingertip according to interference with a geometric 
model of this sort and other models in virtual space. If a glove etc. are used at this time, in order 
to restrain a motion of a hand greatly, in this example, the operation input means (refer to 
drawing 6 thru/or drawing 8) which restrains only the back of a fingertip and a hand is used. 
[0029] Drawing 4 is a flow chart which shows the outline of the geometric model creation 
processing by this example. In this example, sensing of the position of the fingertip of the 
position and posture of the back of a hand, and the thumb, an index finger and the middle finger 
is performed first (Step S1). In the link coordinate system of the hand shown in drawing 5, the 
coordinates of the white round head (sigma 1 thru/or sigma 3 ) of a tip part are measured, and, on 
the other hand, it does not measure about sigma 4 and sigma 5 which are black dots. And since 
only sensor data cannot perform modeling to the flexibility of each joint, the following constraint 
can be added (Step S2). As a constraint, first, it fixes, and two joints of the thumb are root 
(circumference of a Z- ^ -axis )-accepted, and rotate. Next, the angle of rotation of each three 
joint of an index finger and the middle finger is equal. And the third finger and a digitus minimus 
follow operation of the middle finger. 

[0030]According to these constraints, the locus of the fingertip on a link coordinate system 
becomes settled. Then, a joint angle is determined using one ingredient of the sensor data (x, y, 
z) of a fingertip (step S4, approximation shape calculation function). Parallel translation of the 
two ingredients from which a fingertip differs is carried out, and they are amended. The position 
of each joint is also proportioned in the distance from the root of a finger, and parallel translation 
is carried out (Step S5). A polygon is stretched and displayed on the surroundings of each link 
cylindrical. 

[0031] Next, the composition which gives reaction force to a fingertip using the operation input 
means 2A and this is explained. Drawing 6 is the operating grip 21 A holding the fingertip of the 
thumb, an index finger, and the middle finger. The operating grip 21 A is provided with the cap 
member 211 in which the tip part of a finger is inserted, and the two arm members 212,213 
formed in the approximate circle arc of 90 angles which form what is called a gimbal mechanism. 
And the end part engages with the cap member 211 centering on a perpendicular direction, 
enabling free rotation, and, as for the arm member 212, the other end is engaging with the end 
part of the arm member 213 centering on a horizontal direction, enabling free rotation. An 
engaging end with the arm member 212 and a rotating shaft with the arm member 212 and the 
end of an opposite hand cross at right angles, and the arm member 213 is connected with the tip 
part of the link member 21A centering on the vertical direction, enabling free rotation. 
[0032]By taking this composition, rotation of the cap member 211 is attained centering on three 
directions with which receive the link member 25A and directions and a gap also cross at right 
angles, and irrespective of the angle of gradient of the link member 25A, where the fingertip of an 
operator is turned in the arbitrary directions, it can be held. The cap member 211 is good also as 
a ring member which inserts a fingertip only in addition to the composition shown in drawing 6, 
for example. 

[0033] Drawing 7 is a perspective view showing the composition of the modeling device by this 
example which has the function to give reaction force to a fingertip. The operation input means 
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which give predetermined flexibility, and three operating grips are supported in this example, 
respectively, and measures the position and posture of a shell of a hand, It has the input means 
movement energizing mechanism 4B which reduces the inertia force by the mass of an operation 
input means by moving this operation input means itself according to the position of the hand of 
an operator. The operation input means 4B and the input means movement energizing 
mechanism 4B are provided with the drive-motor group for giving reaction force and moving an 
operation input means, the encoder group which measures angle of rotation of a link, etc., etc. 
The input means movement energizing mechanism 4B is provided with the direct-acting type link 
member 44B in order to move an operation input means not only rotational movement but within 
a predetermined radius. Operation of this direct-acting type link member 44 is detected by the 
direct-acting type potentiometer. 

[0034]The arithmetic processing unit 1010 which the coordinates of a fingertip are computed 
based on outputs, such as each encoder of the operation input means 2A and an input means 
movement energizing mechanism, and generates the model data of a hand further in this 
example, It has the display 1001 which displays the geometric model data of the hand generated 
by this arithmetic processing unit 1010 with other virtual bodies. The modeling device is provided 
with the control means 3B which controls operation of an operation input means and the input 
means movement energizing mechanism 4B. The three-dimensional manipulator 10B is 
constituted by the operation input means 2A, the input means movement energizing mechanism 
4B, and the control means 3B. 

[0035]Drawing 8 is a block diagram showing the composition of the control system of the three- 
dimensional manipulator 10B shown in drawing 7. The input coordinate calculation part 31 which 
computes the input position coordinates of each operating grip 21 A of every based on the 
direct-acting displacement signal detected by the detection angle signal from which the control 
means 3B is detected by each encoder, and the potentiometer, It has the motion-control part 32 
which outputs the operating command signal based on this input position coordinate data to the 
input means movement energizing mechanism 4B, and the reaction force generating control 
section 33 which inputs a drive command signal into the drive-motor group of an operation input 
means based on the reaction force data outputted from the arithmetic processing unit 1010. 
[0036] If angle of rotation of each rotation joint 2Aa of the alter operation means 2A, 2Ab, and 
2Ac becomes clear, the position of the operating grip which makes rotation joint 2Ac a reference 
position can be found. In order to change this coordinate system into the coordinate system 
centering on a model object, a homogeneous transformation procession is used. If the coordinate 
value in the coordinate system of each alter operation means 2A of every is changed into the 
coordinate value in a model object, it will become a coordinate value of the tip part for 
generating a geometric model. Also about inclination by the position and normal coordinate of a 
shell of a hand, it computes from the angle of rotation of the rotation joint of the operation input 
means 2A. 

[0037]lf this coordinate value is obtained, the arithmetic processing means 1010 will generate 
the model of a finger using a constraint. Drawing 9 is an explanatory view showing the example 
which displayed the geometric model data generated by this example on the display. By using the 
position information on the back of a fingertip and a hand, and a predetermined constraint, the 
shape of a natural hand as shown in drawing 9 is generable. Hereafter, based on the link 
coordinate system (here, a left hand is illustrated) of the hand shown in drawing 5, the constraint 
and the calculation technique at the time of generating this model are explained. 
[0038] First, the posture of the back of a hand is searched for by making the position of the back 
of a hand into the starting point. Coordinate value sigma 1 of the tip part of the thumb, an index 
finger, and the middle finger, sigma 2 , and sigma 3 are computed. 

[0039]The homogeneous transformation procession in the link coordinate system of the thumb is 
shown in a formula (1). Here (O ir 0 12 , and 0 13 ) f The offset coordinate value of the starting 
point of a axis, C j( and Sj costheta,, sintheta,, As for C^ and Sy, cos (thetaj+thetaj), sin 
(thetaj+thetaj), C 123 , and S 123 express cos (theta 1 +theta 2 +theta 3 ) and sin (theta 1 +theta 2 +theta 3 ). 
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thetaj expresses the angle of rotation of link of the circumference of a l fi axis. Thereby, the 
coordinates of the fingertip of the thumb are shown by the following formula (2). 
[0040] 
[Equation 1] 
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[0041] A following formula (3) shows the homogeneous transformation procession in each link 
coordinate system of an index finger, the middle finger, the third finger, and a digitus minimus. 
Here, y is a subscript showing a finger and 3 and the third finger are [ an index finger / 4 and the 
digitus minimus of 2 and the middle finger ] 5. (O y1 , O y2 , and O y3 ) The offset coordinate value of 
the starting point of a Z y1 axis, costhetaj, sintheta jt Cy, and Sy Cj and S f cos (thetaj+thetaj), As 
for sin (thetaj+thetaj), C 123 , and S 123 , cos (theta 1 +theta 2 +theta 3 ), sin (theta 1 +theta 2 +theta 3 ), C y , 
and S y express cos (alpha y ) and sin (alpha y ). thetaj expresses the angle of rotation of link L yj of 
the circumference of a Z yi axis. Thereby, the coordinates of the fingertip of an index finger, the 
middle finger, the third finger, and a digitus minimus are shown by the following formula (4). 
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[0042] 
[Equation 2] 
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[0043] Next, the equation which asks for the joint angle of the thumb is explained. If the 
coordinates of the fingertip of the thumb obtained by a sensor are made into (P 1X , P 1Y , P 1Z ), the 
relation of a following formula (5) will be obtained from a formula (2). Here, the coordinates of the 
left side express the point on the link coordinate system assumed suitably, and the solution with 
which all of three formulas of the left side are filled does not exist probably. That a solution 
certainly exists is a case where the state of a finger is searched for only using one of the 
coordinates acquired by the sensor. For the purpose, it is necessary to set a state variable to 
one. Then, the following constraints are added in order to search for the joint angle of the thumb. 

[0044]lt fixes and the joint angle of the circumference of the 2nd constraint Z 12 and a Z 13 axis 
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gives flexibility only to the circumference of a axis. The equation of this 2nd constraint and 
the 1st equation to the equation (6) of an equation (5) is obtained. Here, a v a 2 , and a 3 are 
constants shown in a formula (7). A formula (8) will be obtained, if a formula (6) is made into 
a 1 c 1 +a 3 = -a 2 s 1( both sides are squared and it arranges about c v A formula (9) will be obtained if 
d is calculated from a formula (8). Considering the movable range of the thumb, the state of the 
thumb in the assumed link coordinate system becomes settled uniquely from a formula (9). 
[0045] 
[Equation 3] 

| -Ln{j%t m + fa**) - LMjtdi - \ciH) - + 1 [ P IX 
| tuC^ca - Imts) h Lni-fcot - fa) + On \ - ! Piv 

L fi3(75*;» + fa*a) + &u(^|Sm - 5*1*1) +• U\*\ + Oia 



a 3 = Ax - Ox 1 



(a?-a5)c? + 2a l a 3Cl f(a3^a5) = 0 



, : _ _ _____ 

[0046]The equation which asks for the joint angle of an index finger is explained. If the fingertip 
coordinates of the index finger obtained by a sensor are made into (P 2X , P 2Y , P 2Z ), the relation 
shown in a formula (10) will be obtained from a formula (4). In order to draw the equation with 
which a solution certainly exists like the case of the thumb, we decided to add the following 
constraints. 
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[0047]The joint angle of the circumference of a 1st constraint Z 2V Z 22 , and Z 23 axis is equal. If 
costheta^ costheta 2 = costheta 3 = c, the equation of an equation (11) will be obtained from this 
1st constraint from the 2nd equation of an equation (10). Here, a v a 2 , and a 3 are constants 
shown in a formula (12). If the movable range of an index finger is considered and this 3rd 
equation is solved, a solution will serve as an equation (13) and will be given to a meaning. Here, 
A 1 and A 2 are constants shown by a formula (14). This relation is the same also about the middle 
finger. 
[0048] 
[Equation 4] 

[ L Zi c m - L H c ls - L 3l c-} ¥(h x ] ffkxl 
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[0049]By asking for a joint angle from the position coordinate of a fingertip using this formula (8) 
and (13), the geometric model of a hand can be created in comparatively few measure points. 
That is, in order not to restrain operation of a hand or to make detection of a position easy, the 
modeling data of the shape of a natural hand as shown in drawing 9 can be created by the tip 
part of a fingertip, the position of a shell, and detection of only a posture. 
[0050]As shown in drawing 7, in order to detect only the fingertip position of three points, the 
middle finger is made to follow about the positions and shape of a third finger model and a 
digitus-minimus model in this example using the 3rd constraint. That is, all the joint angles of the 
joint of a digitus minimus and the joint of the third finger compute that it is the same as that of 
the joint angle of the middle finger. 
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[0051 ]lf the link coordinates for every finger become settled with the technique mentioned 
above, the cylinder side which makes a medial axis the link shown in drawing 5 is formed, it will 
be giving a hemisphere about a fingertip and finger model data will be generated. It is made to 
rotate about the portion of the back of a hand to the posture which measured the shape defined 
beforehand. The geometric model data of a hand is generated by computing the sum of each of 
this finger model and the model of the portion of the back of a hand. 
[0052] 

[Effect of the InventionJIt is that a geometric model generation part uses the constraint stored 
in this restraint condition storage part according to this since this invention is constituted as 
mentioned above and functions, In order to generate the geometric model data of the 
complicated model object which has a joint by little position information, Since the geometric 
model data of the complicated shape which has a joint based on the position information only on 
the tip part which does not restrict operation of a model object can be generated and load is not 
given to each joint of the model object at all, By being able to make natural operation continue, 
measuring position information with a predetermined sampling period, and generating geometric 
model data. The outstanding modeling device which is not in the former that the geometric model 
according to operation of the model object can be generated, for example, it can input into a 
display etc. can be provided. 



[Translation done.] 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

LThis document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3.ln the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 
[Brief Description of the Drawings] 

[Drawing 1]lt is a block diagram showing the composition of the embodiment of this invention. 
[Drawing 2]lt is a block diagram showing the detailed composition of the restraint condition 
storage part and geometric model generation part which were shown in drawing 1. 
[Drawing 3] It is a block diagram showing the composition of the hardware resources of the 
modeling device shown in drawing 1. 

[Drawing 4]lt is a flow chart which shows the outline which generates the model data of a hand 
according to one working example of this invention. 

[Drawing 5] It is an explanatory view showing the link coordinate system of the hand used by this 
example. 

[Drawing 6] It is a perspective view showing the composition of the operating grip used by this 
example. 

[Drawing 7]lt is a perspective view showing the composition of an alter operation means to use it 
by this example. 

[Drawing 8]lt is a block diagram showing the composition of the control system of the three- 
dimensional manipulator shown in drawing 7. 

[Drawing 9]lt is an explanatory view showing the display example of the geometric model data of 
the hand generated by this example. 
[Description of Notations] 

2A Position measurement means (operation input means) 

23 Imaging camera 

24 Lighting Sub-Division 
1001 Display 

1011 Model data output means (display l/F) 

1012 Restraint condition storage part (for example, hard disk) 

1013 geometric model generation part (for example, CPU) 

1014 geometric model synchronizer (for example, CPU) 

1015 Model data generating means 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

LThis document has been translated by computer. So the translation may not reflect the oriqinal 
precisely. 

2.**** shows the word which can not be translated. 
3.ln the drawings, any words are not translated. 

DRAWINGS 

[Drawing 1] 




[Drawing 3] 
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[Drawing 6] 
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[Drawing 9] 




[Translation done.] 
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